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ABSTRACT 

To address the issues raised by the possible 
expansion of administrator competency testing, a survey was sent to 
each state education agency in late 1988. Using survey data and a 
literature review, this paper discusses the development of 
administrator competency testing, explores pertinent issues, makes 
recommendations for policy makers opting to use these tests, and 
suggests workable alternatives. Currently, 15 states use 
administrator competency tests, so far, there is scant evidence that 
such testing has achieved its primary objective — ensuring that 
certification candidates have attained the knowledge required to 
fulfill the duties of a school administrator. Various issues 
associated with student and teacher competency testing programs are 
relevant to administrator competency testing. These include test 
validity, impact on preparation programs, minority recruitment, legal 
challenges, and costs. Alternative practices being used in several 
states include assessment centers, on-the-job training and 
assessment, on-the-job assessment without training, alternative forms 
of testing, multiple competency indicators, better recruitment 
stiategies, and improved administrator preparation. Policy makers 
need to clarify objectives to be accomplished through testing, 
determine costs of test development, study other states 1 experiences, 
be aware of legal challenges and possible racial and cultural biases, 
and consider alternative practices. (43 references) (MLH) 
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Administrator Competency Testing: 
Status, Issues, and Policy Considerations 

Fifteen states now require administrators to pass a competency test for initial 
certification. But administrator testing is a source of some controversy. 

by Ulrich C Reitzug 



3n the 1970s student competency tests became a popular 
response to cries for accountability in education. The early 
1 980s saw the competency testing movement expand with 
the development of exams for teachers (Hazi, 1 986). Several 
researchers have observed that competency testing for ad- 
ministrators may be a "third generation" in the competency 
testing movement (Eggington, Jeffries, & Kidd-Knights, 1988; 
Hazi, 1 986). In fact, the National Policy Board for Education- 
al Administration (1 989) recently recommended thedevelop- 
ment of a national certification exam for administrators. 

The possibility of the expansion of administrator com- 
petency testing raises a number of questions. What is the 
current status of administrator testing? What objectives are 
states attempting to accomplish with these tests? What can 
policymakers learn about administratcr competency testing 
from prior experience with student and teacher competency 
testing? What alternatives exist to the use of administrator 
testing? 

To help answer some of these questions, a survey was sent 
to each state eaucation agency in late 1988. Follow-up 
surveys were sent to non -respondents in early 1989. Phone 
calls were placed to all states still not responding, with the 
exception of six states (Delaware, Georgia, New Jersey, South 
Carolina, South Dakota, and Utah), for whom recent infor- 
mation was available from secondary sources. Utilizing sur- 
vey data and a review of competency testing literature, this 
paper addresses the above questions, makes recommenda- 
tions for policymakers opting to use administrator competen- 
cy tests, and suggests alternatives for those opting not to use 
them. 

Current Status 

The figure on page 2 shows the development of ad- 
ministrator competency testing through the 1980s. At the 
beginning of the decade, only 1 state (Georgia) required 
administrators to take competency tests. By 1986, this num- 
ber had risen to 10 states. Although Alabama and New 
Mexico recently dropped their testing requirements, 7 addi- 




tional states have i mplemented administrator testing over the 
last three years, bringing the current total to 15 states. 1 All 
these states require a passing score on the test for initial 
administrative certification. 

Test Development 

States utilizing administrator competency testing develop 
their tests through one of three methods. First, five states use 
the standardized National Teacher Exam specialty area exam 
in administration and supervision (EAS). Additionally, Ohio 
and North Carolina intend to begin using this test in 1 990. 

The second method, used by five states, is a test develop- 
ment process marketed by National Evaluation Systems 
(NES). This process consists of several steps. Initially, NES 
officials organize the knowledge base into content areas and 
write objectives for each area using a variety of materials, 
including university program descriptions and textbooks, 
state statutes, policies and regulations, and professional 
literature. Next, NES conducts an extensive job analysis 
survey, in which the state's practicing administrators and 
administrator trainers rate the importance of each objective 
to real-world administrative duties. Based on this survey, NES 
selects a subset of the initial objectives, writes test items for 
each objective, and conducts a field test. Throughout this 
process, a committee of the state's educators reviews the 
content areas, objectives, and test items, revising where 
necessary. After the test validation process is completed, the 
committee recommends passing scores to the State Board of 
Education, which sets the passing grade (National Evaluation 
Systems, Inc., 1937; Oklahoma State Department of Educa- 
tion, 1986; Texas Education Agency, 1987). 



1 New York and Tennessee were included by Hazi (1986) as 
utilizing administrator competency testing but never implemented 
testing. (Testing was proposed but not approved in those states.) 
Connecticut was incorrectly listed as using administrator competen- 
cy te^mg by Eggington et al. 

Alabama discontinued all competency testing in 1988 due to 
extensive legal challenges mounted against the testing program. New 
Mexico ceased requiring the EAS specialty area exam in 1989. 
However, applicants i nitially applying for administrativecertification 
in New Mexico who have not previously taken the NTE core battery 
must do so. 
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The third test develop- 
ment option, used by three 
stater, is a customized test 
development process in- 
volving practicing ad- 
ministrators, university 
professors, and state depart- 
ment of education officials. 
The process followed is 
similar to that used by NES. 
(See Griffin [1988] for a 
detailed description of the 
process used by Florida.) 

Objectives 

Eleven of the 15 states 
that use administrator com- 
petency testing agree that 
the primary objective of the 
tests is to ensure that cer- 
tification candidates have 
attained the knowledge re- 
quired to successfully fulfill the duties of a school ad- 
ministrator (Reitzug survey, 1 989). Two other states (Missouri 
and Washington) consider the improvement of administrator 
preparation programs through the use of competency testing 
as a major objective. (The remaining two states did not 
respond to the question dealing with objectives.) 

Other analyses of competency testing have listed addition- 
al objectives (see Brandt, 1983; tggington et al., 1988; 
Flippo, 1986; Hazi, 1986; Popham & Kirby, 1987): 

• to identify at an early stage candidates unlikely to succeed 
in completing an entire screening process; 

• to improve the image of schools; 

• to compensate for problems with the selection process; 

• to respond to the public's demand for accountability; 

• to produce change in education. 
Still others have suggested that the real objective of com- 
petency tests is to function as a symbolic and political gesture 
rather than as an actual reform (Ellwein, Glass, & Smith, 
1 988; Salganik, 1 985). Ellwein et al. (1 988) argue that com- 
petency tests are designed to give the appearance of account- 
ability, with elaborate safety nets (test retakes, waivers, etc.) 
to catch those who fail, allowing states "to point to tough 
standards while doing business as usual" (p. 7). Indeed, test 
retakes are permissible in all states. It should be noted that 
courts have made multiple retakes a requirement in order for 
a test to be legally defensible (Goldman, 1 984; Marshall, 
Serow, & McCarthy, 1987; Sendor, 1984). 

Achievement of Objectives 

Only seven states responded to the survey question asking 
the extent to whic h competency testi ng objectives were bei ng 
accomplished. Of these, three states indicated that the tests 
had been too recently implemented to ascertain their effec- 
tiveness; one state listed "widespread acceptance" by the 
educational community and the absence of a court challenge 
as indications that administrator competency testing vas 
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accomplishing its objectives; and three states noted that high 
passing rates for the tests were evidence of their success. 

These responses (and the lack thereof) provide scanty 
evidence that administrator competency tests are achieving 
their objectives. Widespread acceptance and high passing 
rates could simply imply that passing scores are set so low 
that administrators do not feel threatened by the test. Addi- 
tionally, while high passing rates may provide evidence of 
knowledge base mastery, they do not provide evidence that 
knowledge base mastery will result in administrative effec- 
tiveness (see discussion of predictive validity, p. 3). 

Ellwein et al. (1 988) claim that efforts to assess the impact, 
utility, and value of competency tests have been "cursory and 
impressionistic" at best (p. 8)— in sharp contrast to the con- 
siderable time, energy, and money invested in test develop- 
ment. Perhaps, they suggest, this contrast underscores the 
symbolic function— as opposed to the instrumental benefit— 
of competency testing. 

Issues 

A number of issues associated with student and teacher 
competency testing programs are relevant to administrator 
competency testing. These issues include test validity, impact 
on preparation programs, minority recruitment, legality, and 
cost. 

Test Validity 

Numerous researchers have raised issues regarding the 
validity of competency tests (see Haney, 1984; Madaus & 
Pullin, 1987; and Neill & Medina, 1989 for detailed discus- 
sions of test validity issues). Validity issues fall into two 
categories: content validity and predictive validity. 

First, does the competency test reflect the knowledge base 
of the field being tested? Each of the three methods of 
administrator competency test development use accepted 
processes to ensure content validity. The traditionally ac- 
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cepted knowledge base, however, is a reflection of ad- 
ministration in a top-down, bureaucratic structure. 
Numerous recent studies advocate alternatives to this tradi- 
tional bureaucratic structure that provide for 'nore employee 
participation in the leadership and decision-making of the 
organization (e.g., Carnegie Forum on Education and the 
Economy, 1 986; Clark & Meloy, 1 989; Naisbitt & Aburdene, 
1 985; Peters, 1 988; Reitzug, in press). These alteratives are 
likely to require knowledge other than that fou r d in the 
traditional knowledge base, but current tests are unlikely to 
include this knowledge. 

Second, is there a correlation between competency lest 
scores and on-the-job performance? Although administrator 
competency testing is too recent a development to have 
acquired a body of research addressing its predictive validity, 
critics have cited numerous studies of tpacher competency 
testing that have failed to establish predictive validity of 
competency test scores with classroom performance (Dar- 
ling-Hammond & Wise, 1983; Linn, Madaus, & Pedulla, 
1982; Madaus & Pullin, 1987). In addition, few professions 
have been successful in establishing predictive validity for 
their entry and/or licensing exams (Stedman, 1 984). 

Impact on Preparation Programs 

Critics have argued that competency tests will drive the 
curriculum (Flippo, 1986). While only two states listed 
"change in administrator preparation programs" as a desired 
outcome of administrator com- 
petency testing, some degree of 
alignment between the objectives 
of administrator competency tests 
and administrator preparation 
programs is likely. This has certain- 
ly occurred with teacher testing 
programs, as some states condition accreditation of teacher 
education programs on the percentage of students passing 
competency tests. In the case of student competency tests, 
some courts have ruled that schools have a legal obligation 
to prepare students for the test (Marshal I et al ., 1 987; Sendor, 
1 984; Stedman, 1 984). Although no court has issued a similar 
ruling for teacher competency tests, the possibility of such a 
ruling is not farfetched With or without the insistence of 
courts or the prompting of accreditation boards, preparation 
programs are likely to adjust their curricula, since the pass/fail 
rate of graduates on competency tests will be perceived as 
reflecting the quality of the program, 

McCarthy (1989) argues that this is not necessarily 
detrimental if the test covers content necessary for effective- 
ness. However, as noted previously, questions remain about 
the content validity and predictive validity of administrator 
competency tests. In addition, some critics have argued that 
widespread testing may retard rather than advance the inter- 
ests of those being tested, since an inordinate amount of 
instructional time may be spent on rote learning at the 
expense of higher level thinking skills that may be important 
for job performance but that cannot be easily assessed by a 
paper-and-pencil test (Evans, 1985; Greaney & Madaus, 
1985; Haney, 1984). The result may be that instead of 
teaching students how to be gooo administrators, university 
preparation programs will simply teach them how to pass 
tests. 




Minority Recruitment 

The adverse effect of teacher competency testing on 
minority group members is well documented (Flippo, 1 986; 
Cifford, 1986; Kauchak, 1984; Linn et al., 1982; Mercer, 
1983; Olson, 1988; Smith, 1984). Given the dispropor- 
tionately high failure rate of minorities on r^npe^ency tests, 
administrator competency testing might not oi v prevent 
aspiring minority group members from becoming c 'rtified as 
school administrators, but might also discourage tuem from 
considering administration as a career option (Gifford, 1986). 
This trend has already been observed as a response to teac her 
competency testing (Olson, 1 988). 

Legality 

Teacher competency testing has been accompanied by 
legal challenges in many states. Most litigation wolves 
claims that the adverse effect of competency testing on 
members of minority groups violates rights protected by the 
fourteenth amendment's equal protection clause or by Title 
VII of the Civil Rights Act of 1964 (McCarthy, 1989). If the 
state can prove that the test is related to a legitimate employ- 
ment objective (such as improving the quality of the work 
force), is properly validated (as by the job analysis surveys 
conducted by NES), and is not accompanied by dis- 
criminatory motives, then courts have usually upheld the test, 
as has been the case in South Carolina, Texas, and Arkansas. 

Many challenges to teacher test- 
ing have been settled out of court. 
To date, no settlement has banned 
the use of a properly validated test 
to make decisions regarding ad- 
mission to teacher education 
programs, certification, or recer- 
tification, although Alabama recently discontinued its testing 
program in the wake of protracted legal struggles. However, 
courts have required modifications in the test instruments, 
validation process, and scoring procedures For example, the 
state may have to agree to evaluate test items for racial and 
cultural bias, or to offer remediation and retake opportunities 
(McCarthy, 1989). Policymakers considering the implemen- 
tation of administrator competency tests would do well to 
ensure that tests are properly validated, unbiased, and fairly 
scored before they are forced to make these adjustments by 
the courts. 

Costs 

There are numerous costs involved in implementing com- 
petency testing. The method of test development determines, 
to a great extent, initial start-up costs. While neither the 
Educational Testing Service (EAS test) nor the National 
Evaluation Systems (NES) would release financial informa- 
tion, use of a standardized test is undoubtedly !ess expensive 
than development and validation of an original test. Sources 
have cited a range of $45,000 to $ 1 50,000 as the cost of test 
development and validation (Flippo, 1986; Rudner, 1987). 
However, these figures do not provide an accurate estimate 
of actual costs, since they do not include the costs of 
revisions, the costs incurred by the state department of educa- 
tion in managing the testing program, the cost of study guide 
development, or the cost of test administration and scoring. 
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Information provided by the I loncla Department of Educa- 
tion indicates that the University ot West Florida received an 
initial grant of $750,000 to manage the development of a 
customized administrator competency exam 'n that state and 
to conduct test administration for two years Subsequently, 
the university received a second grant of $658,793 to ad- 
minister the test for another two years A third grant of 
$10,000 was also awarded to the university to develop a 
study guide for the exam, bringing the total cost to 
$1 ,418,793 for four years of testing (Merlin Mitchell, personal 
communication, April 5, 1989) 

As this example shows, competency testing is an expensive 
process, Analysts suggest, however, that attempting to cut 
corners by providing too little time or money for implemen- 
tation, planning, pilot testing, or evaluation may result *n a 
complete waste of resources {Anderson & Pipho, 1 984). The 
upshot is that if a state wants a competency test for ad- 
ministrators, it must be willing to pay font. 

Alternatives to Administrator Competency Testing 

Pipho (1985) has noted that "nearly every large education 
reform effort of the past few years has either mandated a new 
form of testing or expanded use of existing testing" {p 19). 
Testing is the traditional route to effectiveness It is an easy, 
familiar, and thus safe response to the call for tough stand- 
ards — a way to evaluate people numerically rather than 
qualitatively Since a research base does not exist to support 
its efficacy {Ellwein et al„ 1988), our over-reliance on it as a 
method of educational reform may bedue to political reasons 
(the pressure to support tough standards) or prcctical reasons 
{we have not discovered better ways of accomplishing our 
objectives). 

What alternatives to competency testing are possible? How 
can reforms facilitate and stimulate rather than mandate? 
Following are alternative practices being used in several 
states, as well as practices policymakers may wish to con- 
sider. 

1 . Assessment centers: Missouri, in addition to a paper- 
and-pencil competency test, requires each administrative 
certification candidate to participate in an assessment center. 
Each candidate is required to pay the fee for the assessment 
{currently $400). The state intends to use assessment results 
not only for screening out unqualified candidates, but also to 
match school needs with the strengths of qualified can- 
didates. While issues regarding assessment centers are 
beyond the scope of this paper, they do provide a more 
holistic approach to evaluation than does a paper-and-penci I 
test 

2. On iiie-job training and assessment: North Carolina has 
developed a Quality Assurance Program including two levels 
of certi ication. To attain the second level of certification, two 
years oi administrative experience are required. During this 
time the new principal works with a mentor principal who is 
also responsible for assessing the novice principal's on-the- 
job mastery of state-developed competencies 

Florida utilizes a similar approach lor its Level II certifica- 
tion. Level II certification candidates must serve for a mini- 
mum of one year as assistant principals, intern principals, or 
interim principals. During this time they must participate in 
a district-developed training program and are evaluated on 
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their demonstration of successful performance of the state- 
developed principal competencies. An issue involved with 
intern/men tor-evaluator relationships is the blurring of the 
lines between formative evaluation (used for professional 
growth) and summative evaluation (used for employment 
decisions). 

3. On-the-job assessment without training: Connecticut 
uses a professional skills assessment by a team of educators, 
including one statewide assessor, but provides no additional 
training for certification candidates 

4. Alternative forms of testing: The Educational Testing 
Service is developing an alternative to a paper-and- pencil 
minimum competency test for teachers ("Educational Test- 
ing," 1 988). This alternative test will include such features as 
computer simulations and interactive video segments. Per- 
haps a similar alternative test could be developed for ad- 
ministrators, 

5 Multiple indicators of competency: Instead of a single 
form of assessment (e.g., the competency test), states could 
use a combination of the suggestions listed above. Multiple 
forms of assessment would increase the validity of the evalua- 
tion (Costa, 1989; Haney & Madaus, 1989; Neill & Medina, 
1 989; Shepherd, 1 989). Shulman (1 987) recommends a com- 
bination of written assessments, assessment center exercises, 
documentation of performance during supervised field ex- 
perience, and direct observation of practice by trained ob- 
servers. 

6. Better Recruitment Strategies: Traditionally, candidates 
for certification self-select; that is, they make a decision to 
obtain administrative certification and then proceed to fulfill 
coursework and testing requirements necessary tc do so 
Competency testing serves only to decrease the size of this 
pool by eliminating the least qualified of these candidates 
while doing nothing to improve the quality of the remaining 
pool (Darling-Hammond & Wise, 1983; Franzosa, 1985) 
More vigorous recruiting efforts tor both minority and non- 
minority candidates would increase the quality and quantity 
of the entire pool of candidates seeking administrative creden- 
tials 

Recruiting high quality minority and non-minority teaching 
candidates would also prove beneficial to the administrative 
pool Wise argues that scholarships, fellowships, and loans 
to prospective educators, higher teacher salaries, and profes- 
sional izat ion of the teacher's role will serve to improve the 
quality of the teaching pool 'cited in Brandt, 1983). These 
measures would have a ripple etfect on the quality of the 
administrative pool since almost all administrator currently 
come from the teaching pool. 
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7. Improved Administrator Preparation: Demographics 
indicate that the school administration work force is "gray- 
ing," with a high percentage of turnover due to retirement 
expected in upcoming years. Thus, the next few years are a 
crucial window of change in which improvements in the 
preparation of administrators can reap consequential 
benefits. Several efforts are underway to improve ad- 
ministrator preparation through higher entry standards, more 
rigorous academic courses, field experiences, and the like 
(see National Policy Board for Educational Administration, 
1989). 

Considerations for Policymakers 

Based on the issuesand problems thai have surfaced during 
the implementation of student and teacher competency test- 
ing programs, considerations and recommendations for 
policymakers follow. 

1. Clarify stated and unstated objectives to be ac- 
complished through administrator competency testing. Are 
objectives political or educational ? Are they symbol ic or real ? 
Are administrator competency tests a viable way of ac- 
complishing desired objectives? 

2. Determine the costs of developing and implementing 
administrator competency testing as compared to the costs 
of other strategies tc upgrade the quality of school ad- 
ministrators. From what programs is the money being 
diverted to pay for testing? Are there alternative, more effec- 
tive uses for money being spent on administrator competency 
testing? 

3. Study the impact of administrator competency testing in 
states that have adopted such mandates before investing 
money in the test development proems. How are classrooms 
and schools affected by results of competency testing? How 
do various stakeholders (e.g., teachers, parents, community, 
other administrators) lose or gain by the imposition of com- 
petency testing (Ellwein et al., 1988)? Are the tests screening 
out unqualified candidates? Are the tests screening out only 
those who are unqualified, or are they also screening out 
qualified candidates who are poor test-takers? 

4. Since legal challenges are likely, make certain that 
administrator competency tests follow legal principles estab- 
lished in previous competency testing cases. Clarify the 
employment objective for giving the test. Develop a valida- 
tion process that demonstrates a relationship between ad- 
ministrator competency tests and the objectives for using the 
tests. Review the test for racial and cultural bias. Work 
collaboratively with universities in establishing curricular 
alignment between university preparation programs and test 
content. 

5. Be realistic about what testing can accomplish. At best, 
competency testing will screen out those unqualified in 
academic skills, but it will not assure the effectiveness of 
those who pass 




6. Consider f ome of the alternatives to administrator com- 
petency testing mentioned above. 

Conclusion 

The student and teacher competency testing movements 
have been fraught with difficulties. Competency testing for 
administrators may experience less difficulty than these pre- 
vious movements by heeding the lessons that they have 
taught us. Nonetheless, policymakers considering the im- 
plementation of administrator competency testing should be 
realistic about what the tests can achieve and aware of their 
limitations. Competency testing should, at best, be used as 
one indicator among many of an individual's competency to 
be a school administrator. 
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